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ABSTRACT 



It is very important, when considering how students from one 
area compare with those from another, to use similar student performance 
data. This is particularly evident when comparing the performance of students 
taking the Scholastic Aptitude Test (SAT) from an area where many students 
take the test to the performance of students from an area from which only a 
small percentage of students participate in the SAT. It is more appropriate 
and useful to use SAT results in a way that is similar for each student and 
which shows whether the student improves from one year to the next. To that 
end, the Noblesville, Indiana, schools used the Preliminary Scholastic 
Aptitude Test (PSAT) and SAT scores for the same student, converting the PSAT 
scores to a score similar to the SAT score. This comparison was more 
meaningful in terms of student growth and class growth. The next step is to 
use student performance data to improve performance. (SLD) 
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David Grissmer, in a recent Indianapolis presentation, focused on the current RAND 
report titled Improving Student Achievement: What State NAEP Test Scores Tell Us. He 
^ talked about the importance of comparing similar student performance data when we want to 

know how students are performing. Let me try to explain what this might mean as we look at 
p student performance data on the Scholastic Aptitude Test (SAT) which was recently released for 

w the high school graduating class of 2000. I will start with an athletic example for this 

comparison. Suppose you and 1 coached football of teams with 50 players on each team. We 
both think the players on our team have more foot speed. We decide to settle the issue by having 
a foot race. The only condition I set is that I want to race my 2 fastest players against 30 of your 
players. If all 30 of your players are not faster than my 2 players, does this mean that my team 
has more foot speed than yours? No. Unfortunately, we use those same kinds of comparisons 
with test scores. Let me give you another example, using the recent SAT for the students at 
Noblesville High School. 

Top 10% of 2000 Graduating Class Better Than Any Other State 

In Indiana, there is always a question about the large number of students taking the SAT. 
Is it fair to compare Indiana with 60% of the students taking the SAT in 2000 with North Dakota 
who has only 4% of the students take the SAT? North Dakota had the highest scores on both the 
Verbal and Math parts of the SAT. I would like to show you how a smaller number of students 
would compare. For Noblesville High School students, we combined the Verbal and Math SAT 
into a single score instead the individual ones for each test because it is easier to show total 
comparisons. From Noblesville High School test results, I used the SAT scores from the top ten 
percent of the 2000 graduating class. These results are shown in the table below compared to the 
results from the states that scored at the following levels 1, 10, 20, 25, 30, and 40. 



lop 


Perce ill 


Com hined Score 


1 


4% 


1,197 


10 


7% 


1,164 


20 


8% 


1,138 


25 


13% 


1,125 
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30 


18% j 


1,111 


40 


30% 


1,087 


National 


44% 


1,019 


NHS 


10% 


1,257 


NHS 


79% ! 


1,005 



Source: The College Board at http://www.collegehoard.org/press/seniorOQZhtml/table3.html 

As you can see from the table, the top 10 percent of the students at Noblesville High 
School scored better than all states shown. It was only after you looked at the scores for about 
half of the states that you find more than 10 percent of the students were tested. Is it a correct 
comparison to say the top 10 percent of the students at Noblesville High School compare better 
than all 50 states? Is this comparison any different than the foot race comparison I made earlier? 
I suspect that you would say these are not proper comparisons. I would agree with you, but I 
would state that this is not any less appropriate than how we are presently using student 
performance data. This brings us back to the original point that David Grissmer made: that in 
order to get proper comparisons you need to use similar data, and I would like to give you a 
better way to use student SAT results so that the playing (or reporting) field is even. 

What SAT results should we be considering? 

We had been very pleased with our SAT results in Noblesville Schools over the last five 
years because we had a steady increase in SAT scores. We had worked very hard to bring about 
this increase by changing the academic offerings at our high school with more demanding 
academic courses and increasing the number of students taking those courses. In 2000, our 
average student scores declined from the previous four years. We decided to look more in depth 
at our results, and we wanted to share that information in terms of way to look at performance 
data that is similar for each student and whether the student improves one year to the next. 



In reviewing our records, we found that we had scores on both the Preliminary Scholastic 
Aptitude Test (PS AT) and the SAT. For the class of 2000, we had 317 take the PS AT as juniors 
(82%) versus the 291 (79%) take the SAT as seniors. Although these two groups were not all the 
same students, there were such a high percentage of students within both testing groups that it 
seemed fair to compare their results. Shown in the table below are the PSAT and SAT results by 
class for the last six years at Noblesville High School. 



% Tested PSAT % Tested SAT 



Class 


% 1 C\i'X- 


Combined 




Combined 


Change 


z Score 


2000 


82% 


937 


79% 


1,005 


+68 


1.64 


1999 


60% 


1,012 


79% 


1,029 


+17 


-1.51 


1998 


61% 


994 


75% 


1,033 


+39 


-0.15 


1997 


53% 


988 


68% 


1,029 


+41 


-0.03 
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1996 


67% 


965 


70% 


1,007 


+42 


0.03 


1995 


47% 


881 


72% 


923 


+42 


0.03 










Mean 


41.5 










Standard 


Deviation 


16.2 





PSAT scores were converted to a score similar to the SAT score by dropping the decimal 
point on the PSAT. For example a 51.9 became at score of 519, this is a practice that school 
counselors have used for years to compare the two scores, and the verbal and math scores were 
combined. Differences were compared by computing a mean and standard deviation for all six 
years, and determining through the use of z score (Chase, 1976) computation the statistical 
difference between the test scores. In looking at the student performance from the PSAT testing 
to the SAT testing, we find that there is an improvement for all six years reviewed. The year that 
we had the highest improvement from PSAT to SAT was also the year (2000) that we had the 
least amount of growth (-1.51) from the class tested the previous year. The year in which we had 
the least amount of growth from PSAT to SAT (1999) was a year in which we had our second 
highest total SAT scores (1,029). According to The College Board 2000, on the average juniors 
taking the PSAT in October had a gain of 10 points higher in verbal and 12 points higher in math 
('http://www.collegeboard.org/sat/cbssenior/html/statOOe.html) . The 68-point gain by the Class 
of 2000 was more than three times the national gain as compared to 17-point gain of the Class 
1999. Now those scores gave different meaning because we were looking at similar information 
overtime. Being able to understand and compare similar performance data for the same students 
is a first step if we are ever going to set expectations to improve student performance. 

How should we use the test results? 

The next step is being able to use the student performance data to improve performance, 
not just report it. There are number of things that can be done and some of these things are: 

• identify particular math and verbal skills where students can improve based upon PSAT 
results; 

• implement interventions between the PSAT and SAT to improve the specific math and verbal 
skill which were identified; and, 

• analyze SAT results to make sure that the interventions are achieving the improvement in the 
skill desired. 

We believe that steps outlined above will allow us to develop more specific interventions, 
which will help to improve the performance of all students. If we do this, then is it really 
important whether SAT scores of this class are as high as the previous class? No. Our goal for 
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the information is to convince people that looking at similar student performance data over time 
is the most important thing that we can do to achieve the types of improvement in student 
performance which we all desire. 



Conclusion 

At the presentation by Dr. Grissom, one of the questions was how do all parties involved 
work together for the improvement of student performance. To answer that question, we must 
agree on how we measure student performance. There is not a disagreement about the need for 
students needing greater skills to be successful in the future, but there is considerable criticism 
that the students graduating from Indiana high schools do not measure up with their counterparts 
from other states. It is important that we do more to improve student performance, and agree 
how to measure it, then we will know whether we are seeing improvement or not. Furthermore, 
it is important than we expect all students to improve not just some. If students came to school 
with the same ability, parental support, motivation, and the other external factors which affect 
how they perform, then it would be fair to compare students as if they were the same and the 
only factor that made a difference was the school in which they were enrolled. However, seldom 
are all of those factors equal. 

As we talk about having Indiana student performance measure up and create within the 
individual students the kind of skills that they need to be successful in the future, there are many 
things that must be done. The first thing that must be addressed is the setting of expectations. 

As we have tried to show you here, the expectations shouldn’t focus on just wanting to be able to 
say, “my students are better than yours,” but show whether they are truly improving over time. 
The only way this can happen will be for a change in thinking from rank ordering and comparing 
to looking at student improvement. If improvement becomes our measure of student success, 
then we believe you will see better results in the overall performance of all students. 
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